Assessment

Assessment is an integral element in the student learning process. In this chapter the purposes, types and principles of assessment are discussed. Finally, ways of identifying and combating plagiarism and collaboration between students are discussed.

The purposes of assessment

Assessment can serve many purposes, some of which are:

• judging mastery of essential skills and knowledge
• measuring improvement over time
• diagnosing student difficulties
• providing feedback to students
• evaluating the effectiveness of the course
• motivating students to study

There is a wide range of assessment methods available. Most achieve only one or a limited number of the above purposes. It is important that the method chosen is appropriate to the purpose.

Definitions

Assessments are of two broad kinds: formative and summative.

A formative assessment is a piece of work to obtain an estimate of achievement, and used to help the learning process. It is often in the form of coursework and always includes an opportunity for discussion and/or feedback on performance between the student and the tutor. A formative assessment is for the benefit of the student in terms of guiding their study.

A summative assessment is a piece of work, usually an end of course assessment, producing a measure which sums up a student’s achievement and serves no other purpose than as a description of what has been achieved. Such an assessment contributes to decisions on the student’s future.

However, formative and summative assessments are really the ends of a continuum, along which the two types of assessment may merge.

Integration with other law subjects

As was discussed in chapter 3, total integration of legal research skills within the rest of the law programme is the ideal. In my experience, in both undergraduate and postgraduate programmes, integration of assessments in legal research is closest where the choice of topics on which the assessment is based is related to work in other knowledge areas. It is also found where the principles of information literacy have been incorporated within the learning and assessment outcomes for the course.

It is valuable, especially in vocational courses, to attempt to reproduce the conditions a trainee will find when in practice and to set legal research assessments in subjects totally unrelated to any work the student may have attempted in both their undergraduate or postgraduate studies. Examples of appropriate problems can be found in areas of law such as medical negligence, trade marks, commercial mortgages and local authority administrative law.

It is important that in the marking scheme a specific allocation is made for performance in the three elements of legal research skills – problem identification and analysis, information search and retrieval, and effective communication of results. Otherwise, assessment of the skill becomes submerged within assessment of the knowledge of the law.

If the research skills mark forms a percentage of the overall mark for a piece of work, it is theoretically possible to require students to pass both the legal research element and the knowledge element in order to pass the assessment. In practice this is not recommended, as it presents problems for markers should a student clearly pass either the skill or the knowledge component but fail the other – what is their overall grade: pass or fail? Keep the marking scheme simple.

Another variation which should be avoided is to set two or more pieces of legal research as counting towards a single assessment simultaneously. This might be thought educationally sound, in that it provides an opportunity to test the application of the skill in several different knowledge areas. The problem comes at the marking stage – if a student is weak or very weak, even to the point of a fail, in one piece of work, how does that affect the overall assessment mark. Can the other, stronger, piece of work compensate?

In vocational courses it is usual to link assessment of the skill to one of the knowledge subjects. In the case of the Legal Practice Course the topic of the problem to be solved might be based in Business Law and Practice or Litigation. In the Bar Vocational Course (the BVC) the current specification guidelines (General Council of the Bar, 2004) treat legal research as pervasive. They require the skill to be separately assessed, and, in addition, assessments of some other skills and the knowledge subjects must be designed to take into account students’ competence in legal research. Several BVC providers have chosen to couple the main legal research assessment to another skill, usually opinion writing. Another provider has decided that the legal research assessment should stand alone.

A further aspect of the integration of assessment must be considered – if the legal research skills course has been taught through a partnership between staff of the teaching department and service departments (librarian and/or computer staff), then it follows that assessment should be a partnership also. There is perhaps less of a case for involving IT staff in the process than for involving library staff, since the competency of students in aspects taught by IT staff will be evidenced in the standard of presentation of written work. Increasingly, assessments and even exercises are required to be submitted in wordprocessed form. The assessment of the standard of presentation ought to be within the capabilities of either library or teaching staff. However, the librarian can also provide particular input on the effectiveness and comprehensiveness of the research trail, the sources searched by the student, the currency of the information cited, and the content and layout of the list of references at the end of the work.

Assessment and library resources

It is important that the person setting any form of assessment requiring the use of library resources carefully considers the likely impact of the cohort of students descending on a limited range or materials, suite of PCs or printing facilities. Competition at the shelves or in the PC area can be avoided if sufficient time is allocated for the research assessment.

On vocational courses, some tutors try to replicate the time pressure for legal research a trainee solicitor or pupil barrister may have to work under when eventually out in practice. I have come across time limits of just a few hours or a day within which students are expected to research a problem and submit a research trail. Students may be permitted several further days to write up the research findings into the form of a memorandum. As admirable as this mirror of practice might be, such an extreme form of time restriction can place a strain on limited library resources and should be employed only following the fullest discussions and closest collaboration between teaching and library staff.

The principles of assessment

Effective assessment depends on three principles – reliability, validity and explicitness.

Reliability refers to accuracy and repeatability of assessment. It is associated with the processes surrounding assessment, from assessment planning through submission to marking and on to exam boards and transcripts.

Validity refers to whether the assessment assesses what it is supposed to. It is associated with alignment of assessment to learning outcomes, range and volume of assessment.

Explicitness refers to the clarity of assessment to all involved in the process. It is associated with the quality, quantity and timeliness of information given to staff and students regarding assessment.

Reliability: marking

Ideally, assessed work and exercises should be marked by the library staff who set them and undertaken in co-operation with relevant academic staff. Markers should discuss and agree their approach before commencing any marking.

A key component of assessment is consistency of marking. This process is assisted by setting down the criteria for marking and using a standard marking sheet. There are several different designs for the marking sheet.

An unstructured marking sheet (illustration 17 – available below) provides the marker with an opportunity for free comment but can lead to ‘impressionistic marking’. Additionally, it may not adequately address the principle of validity in assessment by not explicitly ‘linking back’ to the learning outcomes for the course. However, it may be the best sheet to use when marking reflective or essay type of work.

A semi-structured scheme (illustration 18 and illustration 19 – available below) sets out the criteria a marker should observe. It is important to ensure that the criteria are being allocated and employed effectively. The two examples show different ways of presenting this. The latter is based on the actual learning outcomes for the Practical Legal Research module on the Legal Practice Course.

A fully structured marking scheme (illustration 20 – available below) allocates a portion of marks to each of the criteria to be considered by the marker. It is important to ensure that the marks for each criterion are being allocated effectively. The example relates to the set of research trail questions shown in illustration 23 – available below.

Care should be taken with the semi-structured and structured schemes so as not to treat them as merely ‘tick box’ marking, but that qualitative matters such as, for example, ‘coherency of argument’ are fully considered.

Where an exercise requires one word or short answers, the marker should annotate incorrect answers with the correct response and include a note on the best method of achieving the correct answer. It is good practice to prepare marker’s notes, which set out the correct answers and provide information on how the correct answer has been obtained. Such notes will ensure consistency in the marking and annotation of student exercises, both for a single marker, and, more especially, where several staff working independently are marking the same exercise. An example of such notes is given in illustration 21 – available below. These have been used for several years by at least three markers marking over 350 scripts each year (the average annual first year intake to the LLB course at Cardiff University).

The Freedom of Information Act 2000 gives a general right of access to information held by public authorities. At some universities students have asked law schools to provide them with copies of assessments, feedback sheets and marking sheets used in conjunction with the assessment process. The request is usually prompted by a student considering the preparation of an appeal. Care should be taken in what markers write on any assessment related papers about the quality of the answer. Comments should be positive and give appropriate, constructive feedback.

Reliability: feedback

It is good practice to provide constructive feedback on student performance. In addition to marking, another means of providing feedback is for the tutor to prepare a model answer. For short answer exercises the model should cover the actual work set and is best distributed to students when their submissions are returned. Illustration 22 (available below) shows the set of model answers given to students when their workbooks on the material in illustration 6, illustration 7 and illustration 8 (available below) were returned.

For long answer exercises, where there may be several different yet equally correct ways of answering the question, a model exercise is best used as a pattern for what the tutor expects in terms of content and presentation. The model should be handed out before or at the same time as the exercise is set and, obviously, should be on a topic unrelated to the actual exercise.

Validity: forms of assessment

It is always good practice, but especially true where the principles of information literacy have been fully embedded within a course, to ‘link back’ the form and content of the assessment to the learning outcomes for the course as a whole. Often these will provide an indication of which activities a student should undertake and how they should be assessed.

There are several ways in which legal research skills may be assessed, but it is important, as mentioned in the first section of this chapter, to ensure the method chosen suits the purpose of the assessment.

Assessment methods suitable for legal research include computer-aided learning through, for example, a quiz at the end of an online tutorial, compilation of a case note (preferably on a recently decided case which has not been mentioned in textbooks yet), an essay on a topic of recent or current interest, compilation of a critical or annotated bibliography, compilation of a literature review, compilation of a portfolio of work with a critical commentary on the research skills developed and displayed, and finally, problem-based research by an individual or a group of students.

Most of these methods are well known and used, so in the following sections just three are described – problem-based research, the concept of the research trail (which can be attached to any form of written work) and group work.

Problem-based research: focused and unfocused problems

A legal research problem may be either focused or unfocused.

A focused problem contains a clearly expressed request for a particular piece of information, such as what is the penalty for a particular offence, the period of notice, the appropriate form, the relevant public organisation, the limitation period, etc. More complex but still focused is the request for the authority of a well known piece of information, such as the drink-drive limit, that seat belts must be worn in motor vehicles, the minimum temperature for places of work.

Generally speaking, a focused problem requires the student to classify the problem and commence a search through appropriate sources – the answer may be elusive, but it will be factual and either right or wrong.

Focused problems are best used in exercises when students are learning how to use a paper or electronic source for the first time. By keeping the law content simple students can concentrate on understanding and assimilating the complexities of learning a new skill. Virtually all the problems featured in the workbooks from which extracts have been taken in illustration 4, illustration 6, illustration 7 and illustration 8 (available below) are of the focused variety.

An unfocused problem is a better test of a lawyer’s research skill. From a mass of information – some relevant, some irrelevant – the student lawyer has to recognise and select the significant facts to arrive at the legal issue/s. The appropriate legal rules need to be sought and applied to those facts to provide ‘advice’ for the client. Unfocused problems can be devised to have definite and ascertainable answers, but, in vocational courses especially, it is important to model the problems more closely on real life, with facts and issues for which, in part at least, there is no clear answer or for which students need to identify which further and better particulars need to be obtained from the client.

Unfocused problems are suitable for assessed work, set after students have learnt basic search techniques and are ready to tackle problems similar to those they will face in legal practice.

Research trails

There are several types of research trail, and they can be used equally well in all types of course – undergraduate, postgraduate and vocational. As far as the principles of information literacy are concerned, the type that fits the criteria best is the ‘reflective’ research trail. Ideally, the trail requires the student to compile reflections on the adequacy and usefulness of the sources consulted in the preparation of a piece of written work. It supplements and does not replace the list of references or bibliography at the end of the written work.

A typical research trail might require the student to set out the research plan or strategy they adopted, list the full bibliographic details of all the items identified as being relevant to the research (the list might include material not eventually cited in the written work), outline the steps taken to uncover the relevant information and list the keywords selected to find information. The final section of the research trail is the important part – students should be asked to reflect on how useful they found each item listed, according to various qualities such as relevance, reliability, authority and objectivity. Additionally, the trail might ask: “if you were asked to undertake the research process again what might you change next time?”. Illustration 23 (available below) shows an example of a ‘reflective’ research trail pro forma issued to students. This type of research trail is appropriate for use in the early stages of a course of study, to awaken and develop critical thinking about sources.

Another form of research trail places less stress on ‘reflection’ but more on the student reporting on the process of selecting appropriate keywords and providing a step by step account of how a particular database or paper publication was accessed and relevant information extracted. Illustration 24 (available below) is an example of a ‘process focused’ research trail.

This type of research trail may be better suited to a point in a course where a student is required to simply focus on problem identification (selecting appropriate keywords) and correctly and accurately undertaking the research process in particular sources. It can be used with both focused and unfocused questions, although illustration 24 shows its use with the former.

Group work

In many institutions legal research exercises and formative assessments (see below) are designed for students to undertake working individually and to a relatively generous time limit (as compared with the pressures they will face in working life). At Cardiff University tutors and library staff teaching on the Legal Practice Course adopted an alternative approach involving group research and preparation of the answer to the formative assessment. Unfortunately, with a change in staff and the need to standardise assessment procedures across all subjects the innovation ceased, but a brief description is given here since some interest outside Cardiff was shown in the experiment.

For several years in the late 1990s the whole cohort of 128 Legal Practice Course students was divided into small groups of no more than six students. Each group was allocated a problem from a selection of three and required to research it and present a response in the form of a legal memorandum to the partner of a law firm together with a letter of advice to the client. Each group was allocated just three weekdays from receiving the instructions to submitting the practice responses, while other coursework and teaching continued as normal. Groups received instructions staggered over a 10 day period to avoid competition at the shelves and computer terminals. In addition, each group was warned not to communicate with any other. As well as the responses outlined, each group was required to prepare a report describing the operation of the group and appraising their success in meeting the goals set. The report covered topics such as how many meetings took place, what was decided, how tasks were allocated and to whom, details of the research trails, the sources searched and the results obtained, and the time spent on the practice assessment, as well as reflections on whether they would approach the research in a different manner next time.

After the legal skills tutor and the librarian responsible for instruction in legal research skills had marked the student responses students were timetabled to attend viva voce examinations, two groups at a time. The tutor and librarian then questioned the groups on their work, highlighting good practice and drawing out points which would assist students in improving their performance.

Student reaction to the experiment was very positive: they said they:

• found the strict and short time limits a challenge but appreciated the experience of working to conditions similar to those found in practice
• learnt more about the conduct of legal research by working as a group and learning from one another than if they had had to work individually
• found the group viva voce examination a more valuable experience than the usual pattern of individual feedback given on the course
• appreciated learning of the variety of approaches, research trails and sources used by other groups
• understood better how groups do (and occasionally do not) function effectively

Tutors’ reaction was positive also, as it enabled them to:

• test if students really understood the law and advice they had written
• clarify or explain general misunderstandings and concerns amongst students

Since group assessment is not a possibility under existing Law Society specifications for the Legal Practice Course the summative assessment which was set after the formative followed the traditional pattern – students working individually and to a two or three week deadline.

Formative assessments

It is valuable to both tutors and students to provide an opportunity when students can undertake a formative assessment. The purpose of the formative assessment is to expose students to the type of problem they will face in the summative assessment (the work which will count towards their passing or failing the course), to provide both tutors and students with a measure of student progress towards the goal of passing the course and an opportunity to give and receive feedback on performance.

Formative assessments have been employed on the Legal Practice Course (LPC) and the Bar Vocational Course (BVC) at Cardiff University for a number of years, and have proved valuable. They usually take the same form as the summative assessment – LPC students are issued with a memorandum from a partner in a fictitious law firm, giving details of an interview which has taken place with a client or a message received from a client. The partner requests the trainee to undertake research on the problem and provide advice. A typical example is shown in illustration 25 (available below), used in 1996 just following the long delayed implementation of an EC Directive relevant to the answer. The trainee’s response has to be in the form of a legal memorandum.

On the BVC the request for research comes from the pupil master or pupil mistress, and students have to respond in the form of a research memorandum. However, students are required to undertake the research against the clock (one day) and submit a research trail, outlining the sources used and the authorities which will be cited in the memorandum. Three or four days later students are required to submit a research memorandum setting out their advice on the problem. If a student introduces into the memorandum material not mentioned in the research trail, it is ignored by the marker.

On both courses the formative assessments are marked under the same conditions as the summative assessment and students receive individual feedback. But for the formative assessment, unlike the summative assessment, students have their work returned with tutors’ comments noted on them. In choosing topics for both formative and summative assessments every effort is made to select a problem in which the law has changed very recently, and a correct answer cannot be compiled solely by using textbooks but only by using paper current awareness and looseleaf updating services or electronic databases, especially Internet services which are updated daily.

Six matters regularly dominate the feedback to students. A student response to the problem must be:

• compiled as a result of a wide ranging search of both paper and electronic sources – although students may believe to the contrary, the questions set cannot be answered as a result of using only one or two sources
• checked to ensure it contains an up to date statement of the law
• focused on the needs of the client (both expressed and implied)
• presented in a more condensed and highly structured form than the academic essay to which students are used
• checked for spelling, grammar and punctuation
• written in an appropriate and adequate English style

The value of the formative assessment lay in acting as an important transition stage between the undergraduate, academic approach to legal research and the client based, practice approach.

Explicitness

It is good practice to provide students with the marking sheet and discuss its contents with them before the assessment is set to help them understand the criteria for the assessment and provide transparency to the assessment process.

Plagiarism and collaboration

Avoiding plagiarism

Plagiarism can range from omitting to use proper citation conventions to the wholesale ‘lifting’ of material from the work of another author (who could be either a fellow student or the author of material in the public domain). The topic is comprehensively covered by Alison Bone’s guide for UKCLE, Plagiarism: a guide for law lecturers, and its accompanying student guide, All my own work: plagiarism and how to avoid it.

Warnings about plagiarism and the consequences of committing it ought to be included in the student handbook. Additionally, the handbook should set out the methods approved by the teaching department for the content and layout of citations, bibliographies and lists of references. The closest to an official handbook on law citation practice in the UK is the Oxford Standard for Citation of Legal Authorities, a system of citation and presentational style for use in legal writing, covering abbreviation, punctuation, cross-referencing, the use of headings and other topics. It was produced by the Oxford Law Faculty in consultation with leading academic law publishers and serves as the style guide for the University of Oxford’s Commonwealth Law Journal, as well as for theses written in the law faculty.

Judging from the number of queries put by students to library staff just before the date of submission of either their first coursework or their thesis/dissertation, some universities appear not to make this information sufficiently comprehensive. Sometimes a tutor may either over-ride or simply be unaware of the guidance given students and require them to employ a citation style similar to “any of the major law journals, such as Cambridge Law Journal, Law Quarterly Review or Modern Law Review“. This type of direction should be avoided for two reasons:

• each journal adopts a slightly different style in the content and layout of citations and the lists of references, so a comparison of the quality of the work of students is made more difficult
• the ‘style sheets’ or ‘notes for contributors’ are not printed within the journals themselves; the student is left to work out from the footnotes and lists of references in the journal exactly what the conventions are, and devise, as best he/she can, what standards the journal is using and which should be applied to their work to be submitted to the tutor, becoming a matter of hunt the most similar reference to the book, journal, official publication, etc that the student wishes to cite

Identifying plagiarism

The usual clue that material has been ‘lifted’ from another source is a sudden change in the content and style of presentation. Suspicions are also raised if the content of the material is of quite a different quality to that submitted by the same student in past exercises and assessed work. Look particularly at the use of jargon, advanced vocabulary or sentence structure. Proving from where the material has been lifted is more difficult. The mushrooming number of Web databases and sites, some of whose contents change rapidly, and the requirement that students search widely for information, mean that material could be copied from an obscure source without the marker being able to prove it.

Since 2001 the Joint Information Systems Committee (JISC) of the Higher Education Funding Councils has provided a Plagiarism Advisory Service (now Plagiarism Advice), designed to offer advice and guidance and to host the UK implementation of the US registered electronic detection service Turnitin. The service was free to use initially, but from 2005-06 an institutional charge and membership applied.

Identifying collaboration

This difficulty in identifying plagiarism may be balanced by the ease with which it is now possible to identify collaborative work or copying amongst students, since an increasing number of pieces of student work must be submitted in wordprocessed form. Similarities in formatting, details of punctuation and layout all point to one disc or file being the source for work submitted by several students individually. Detailed questioning of individual suspected students on the steps they personally followed in using a particular database or publication (why did you select it, which parts did you use, which search terms did you use, how was the information displayed on the screen, where are the volumes you consulted shelved in the library, which other sources are mentioned in the case/article/commentary you have cited, etc…), plus a request to see any rough notes or research trails, usually uncovers evidence that only one student knows how the research was undertaken and the other/s are clueless.

If suspicions of collaboration between students have been raised but cannot be proved, one method of defeating future repetitions is to set each of the suspected students a completely different legal problem to research and write up.

To aid tracking plagiarism in theses or dissertations tutors should consult published lists. Index to Theses (available in paper and Web formats) is the only comprehensive listing of theses accepted for higher degrees in universities in Great Britain and Ireland and by the Council for National Academic Awards, since 1951. Dissertation Abstracts International (in paper form) or ProQuest Digital Dissertations (Web format) lists doctoral theses accepted by over 400 United States and Canadian degree awarding institutions since 1861.

Summary

It is important to select the most appropriate method of assessment to match the purpose. Effective assessment depends on three principles – reliability, validity and explicitness. Research trails, particularly the reflective research trail, apply aspects of the experiential learning theory discussed in chapter 2 and lead to deep learning. One of the purposes of assessment is to evaluate the effectiveness of the course of study – a point explored in more detail in the next chapter.

References

• Bone A (2003) All my own work: plagiarism and how to avoid it (UKCLE teaching resource note) Coventry: UK Centre for Legal Education
• Bone A (2003) Plagiarism: a guide for law lecturers (UKCLE teaching resource note) Coventry: UK Centre for Legal Education
• General Council of the Bar (2004) Bar Vocational Course specification requirements and guidance (revised August) London: General Council of the Bar
• Oxford Law Faculty (2005) Oxford Standard for Citation of Legal Authorities (OSCOLA) Oxford: University of Oxford Faculty of Law

Last Modified: 30 June 2010